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SUMMARY 


Most  jobs  involve  a  variety  of  skills  that  must  be  learned.  However, 
variability  in  job  demands  may  lead  to  periods  when  some  skills  are  not  used 
and  hence  deteriorate.  Little  research  has  been  done  on  this  process  of 
deterioration.  In  this  paper  we  propose  a  number  of  sample  methodologies  for 
investigating  the  retention  of  skills  ranging  from  perceptual  to  motoric  to 
cognitive.  These  methodologies  are  based  on  a  theoretical  analysis  of  the 
existing  literature  on  skill  acquisition.  Research  employing  these 
methodologies  should  allow  for  testing  hypotheses  about  skill  maintenance  and 
should  lead  to  practical  recommendations  for  training  procedures  that  will 
optimize  skill  retention. 


PREFACE 


This  paper  presents  a  set  of  proposed  methodologies  to  use  in  the  study  of 
skill  maintenance.  A  companion  piece  (Fendrich  et  al.»  1988),  separately 
published,  provides  the  background  for  tnis  proposal  in  the  form  of  a  litera¬ 
ture  review  and  theoretical  analysis  of  the  subject.  Together  these  documents 
are  meant  to  provide  the  basis  for  a  research  program  on  skill  maintenance. 
We  wish  to  acknowledge  the  help  of  Antoinette  Gesi  and  Debbie  Aguiar  in  the 
preparation  of  this  paper. 


SKILL  MAINTENANCE:  SPECIFIC  SAMPLE  METHODOLOGIES 


Even  the  most  routine  of  military  and  civilian  jobs  involve  a  variety  of 
component  skills  that  must  be  mastered  by  a  trainee.  Day-to-day  variations  in 
job  demands  dictate  that  the  competent  worker  have  ready  access  to  required 
skills.  Particular  component  skills,  acquired  during  training,  might  not  be 
activated  for  use  for  considerable  time  periods.  Disuse  may  lead  to  skill 
deterioration  which,  in  the  extreme,  might  mean  that  a  skill  is  no  longer 
functional  when  needed.  Procedures  must  be  developed  to  maintain  skills  at 
functional  levels  over  periods  of  disuse.  Although  there  has  been  considerable 
research  designed  to  develop  training  procedures  for  efficient  acquisition  of 
skills,  relatively  little  is  known  about  training  procedures  or  refresher 
techniques  that  lead  to  maintenance  over  time. 

In  this  document,  we  propose  a  number  of  specific  methodologies  for 
investigating  training  conditions  and  refresher  techniques  which  contribute  to 
skill  maintenance.  Our  methodologies  derive  from  a  thorough  review  of  the 
literature  on  skill  analysis  and  on  the  characteristics  of  skill  components 
associated  with  high  maintenance.  These  methodologies  address  a  variety  of 
skills,  ranging  from  perceptual  to  motoric  to  higher-level  cognitive.  These 
methodologies  will  allow  us  to  explore  conditions  of  training  for  a  variety  of 
skills  that  lead  to  efficient  maintenance  and  retrieval  on  demand.  In  the 
following  sections,  we  describe  several  apposite  methodologies,  along  with 
suggestions  for  possible  research.  These  methodologies  will  not  only  allow  us 
to  test  hypotheses  about  skill  maintenance  but  also  should  lead  to  precise 
conclusions  and  practical  recommendations  for  training  procedures. 

Target  detection.  Drawing  cn  methodology  developed  earlier  in  our 
laboratory  to  study  pattern  recognition  processes  in  reading  (see,  e.g.,  Healy 
&  Drewnowski,  1983;  Proctor  &  Healy,  1985)  and  drawing  on  methodology 
previously  developed  by  Shiffrin,  Schneider,  and  their  associates  to  study 
controlled  and  automatic  information  processing  (see,  e.g.,  Schneider  & 
Shiffrin,  1977),  we  have  devised,  with  the  help  of  Richard  M.  Shiffrin  and 
Janet  D.  Proctor,  a  paradigm  in  which  to  investigate  the  long-term  retention  of 
target  detection  skills.  In  this  paradigm,  subjects  are  exposed  to  a 
succession  of  frame  sequences  on  a  CRT  screen  driven  by  a  PDP  11  computer. 
Each  sequence  includes  52  frames,  each  of  which  is  exposed  for  a  fixed  duration 
(1.5  sec).  Every  frame  includes  16  alphanumeric  characters  and  symbols  plus 
two  intermediate  blank  spaces  arrayed  horizontally  to  resemble  three  words  of 
text.  (Actually,  any  of  a  variety  of  stimulus  configurations,  including 
complex  waveforms,  can  be  used.)  Half  of  the  frames  in  every  sequence  of  52 
include  a  target,  and  half  do  not.  There  are  three  frame  sizes,  which  are 
manipulated  by  varying  the  number  of  noise  symbols  (signified  by  #).  Frame 
size  16  includes  16  alphanumeric  (or  other)  characters  and  no  noise  (e.g.,  QXR 
NPLGN  RTCBAIEX) ;  frame  size  4  includes  4  alphanumeric  characters  and  12  noise 
characters  (e.g.,  Q#R  ##L##  #T ######) ;  and  frame  size  2  includes  2  alphanumeric 
characters  and  14  noise  characters  (e.g.,  ##R  #####  #T ######).  Every  block  of 
3  successive  sequences  of  52  frames  includes  one  of  each  frame  size,  in  a 
random  order.  For  subjects  in  the  consistent  mapping  condition,  the 
alphanumeric  characters  are  randomly  selected  letters  and  the  target  is  fixed 
across  blocks  (say  the  letter  H),  whereas  for  subjects  in  the  varied  mapping 
condition,  different  targets  are  used  across  blocks  (so,  for  example,  H  is 
sometimes  the  target  and  sometimes  a  distractor).  Subjects  respond  by  preising 
a  key  whenever  they  detect  a  target,  and  both  their  speed  and  accuracy  are 
recorded  by  the  computer. 
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We  have  devised  three  tests  to  assess  the  degree  of  skill  training 
attained  by  a  subject.  Two  of  these  tests  make  use  of  criteria  developed  by 
Schneider  and  Shiffrin  (1977).  Both  of  these  tests  address  differences  in 
error  rates  and  response  latencies  for  the  three  frame  sices.  Initially,  speed 
and  accuracy  are  steep  functions  of  frame  size,  but  as  training  progresses,  the 
slopes  of  the  functions  decrease  for  the  consistent  mapping  condition,  but  not 
for  the  varied  mapping  condition.  The  first  criterion,  defining  the  attainment 
of  skill  automatism,  is  the  point  at  which  the  slope  in  the  consistent  mapping 
condition  is  significantly  less  steep  than  that  in  the  varied  mapping 
condition.  The  more  stringent  criterion,  defining  the  perceptual  "pop  out" 
phenomenon  (see,  e.g.,  Treisman  &  Paterson,  1984),  is  the  point  at  which  the 
mean  error  rates  and  response  latencies  for  the  three  frame  sizes  are 
approximately  equal.  The  third  test  we  developed  makes  use  of  the  observation 
that  subjects,  in  performing  an  analogous  letter-detection  task  with  text 
presented  on  a  terminal  screen  three  words  at  a  time  (see,  e.g,  Proctor  & 
Healy,  1985),  typically  make  many  more  errors  and  respond  much  more  slowly  when 
a  target  occurs  in  a  highly  familiar  configuration  (like  h  in  the  word  the) 
than  in  a  less  familiar  configuration  (like  h  in  thy) .  This  pattern  of  errors 
has  been  attributed  to  the  subjects'  unitization  of  familiar  visual 
configurations.  In  pilot  work  with  this  paradigm,  we  found  that  after  a  period 
of  training  insufficient  for  the  automatism  criterion  (approximately  4  hours), 
subjects  showed  a  dramatic  change  in  their  pattern  of  errors  in  the 
target-detection  task,  so  that  the  large  difference  between  targets  (e.g., 
letters)  in  common  and  rare  configurations  (e.g.,  words)  was  eliminated. 
Hence,  our  third  training  criterion  will  involve  elimination  of  the  unitization 
effects  in  a  test  sequence  (e.g.,  a  prose  passage). 

Different  groups  of  subjects  will  be  trained  to  each  of  the  three  criteria 
in  an  initial  study.  We  will  then  allow  a  long  (6-month)  retention  interval  to 
elapse,  at  which  point  subjects  will  be  retested  and  retrained.  We  expect  that 
subjects  trained  to  the  less  stringent  unitization  criterion  will  be  less 
likely  than  the  other  groups  to  maintain  their  level  of  performance  across  the 
long  retention  interval  and  that  they  will  no  longer  be  able  to  meet  the 
unitization  criterion  (they  will  once  again  make  more  detection  errors  on 
familiar  configurations  than  on  unfamiliar  configurations  in  the  test  passage). 
However,  if  automatism  does  indeed  lead  to  skill  permastore,  as  proposed  in  our 
first  hypothesis,  then  those  subjects  trained  to  the  automatism  criterion  may 
not  deteriorate  in  performance,  as  assessed  either  by  the  slopes  of  their 
reaction  time  and  error  functions  or  by  their  pattern  of  responses  in  the  test 
prose  passage.  The  "pop  out"  criterion,  representing  a  still  more  stringent 
criterion,  ought  to  lead  to  equivalent  or  possibly  even  superior  skill 
retention. 

The  details  of  the  retraining  portion  of  this  experiment  depend  on  the 
retention  outcome.  Retraining  is  pertinent  only  under  conditions  of  skill 
deterioration.  In  the  retraining  phase,  we  are  primarily  interested  in  the 
savings  afforded  by  the  various  original  training  criteria.  This  phase  will 
also  allow  us,  however,  to  look  at  transfer  effects  remaining  after  periods  of 
disuse  (see,  e.g.,  Rabbitt,  Cumming,  &  Vyas,  1979),  by  examining  performance  on 
targets  that  are  similar  but  not  identical  to  those  used  originally  (e.g.,  h 
instead  of  H  or  T  instead  of  H),  and  at  the  effects  of  retraining  with  only  a 
subset  of  the  targets  originally  trained. 

• 

Mental  calculation  skills.  A  review  of  the  literature  has  shown  that  a 
simple  measure  of  knowledge  retention  at  the  end  of  training  is  not  an  adequate 
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predictor  of  long-term  retention,  either  quantitatively  or  qualitatively.  The 
purpose  of  this  study  is  to  determine  whether  a  measure  of  skill  automatism 
derived  from  the  work  of  Shiffrin  and  Schneider  (1977)  can  aid  in  predicting 
long-term  retention.  The  Shiffrin  and  Schneider  framework  leads  to  the 
definition  of  automatism  as  the  degree  to  which  difficult  operations  can  be 
performed  as  accurately  and  as  quickly  as  simpler  ones,  or  the  slope  of  the 
"functions  denoting  the  relationship  between  accuracy  or  speed  of  solution  and 
problem  difficulty.  In  this  stud'/,  we  will  examine  the  skill  of  mental 
multiplication,  although  this  paradigm  can  easily  be  applied  to  other 
calculational  skills  such  as  base  conversion  or  linear  algebra  problems.  It  is 
predicted  that  an  automatism  index  will  prove  to  be  a  better  predictor  of 
long-term  retention  than  an  index  based  on  overall  performance  levels  such  as 
percent  correct  or  mean  response  latency. 

All  subjects  will  begin  a  training  period  on  solving  multiplication 
problems.  Single-digit  multiplication  problems  will  be  presented  on  a  CRT 
screen,  along  with  an  answer  which  is  either  correct  or  incorrect.  Subjects 
will  respond  by  pressing  one  of  two  buttons  to  indicate  whether  the  solution  is 
correct  or  incorrect.  It  is  predicted  that  initially  subjects  will  be  less 
accurate  and  slower  in  verifying  the  answers  to  problems  with  larger-digit 
operands  than  with  smaller  ones  (the  slope  of  the  performance/difficulty 
functions,  will  be  very  steep).  Based  on  the  theory  of  automatism,  as  the 
training  progresses  subjects'  performance  will  not  only  improve  overall  but  the 
relative  difference  between  the  simple  and  the  difficult  problems  will  decrease 
(the  performance/difficulty  functions  will  become  much  flatter).  After 
training  is  terminated,  an  interval  of  6  months  will  expire,  followed  by  a 
retention  test  in  the  same  paradigm. 

In  the  first  phase  of  this  study,  total  training  time  will  be  fixed  and 
the  final  levels  of  performance,  by  both  overall  and  automatism  standards,  will 
be  compared  in  their  abilities  to  predict  long-term  retention.  It  is  expected 
that  the  automatism  measure  will  be  a  better  predictor  of  retention  than  will 
the  overall  performance  level.  The  data  obtained  from  the  first  phase  will  be 
used  to  estimate  the  level  of  performance  at  the  end  of  training  necessary  to 
retain  the  skill  at  a  given,  high  level.  This  method  is  similar  to  that  used 
by  Bahrick  (1984),  except  that  he  estimated  optimal  study  time  rather  than 
optimal  study  performance.  Optimal  performance  estimates  will  be  obtained  for 
both  overall  performance  levels  and  the  slope  of  the  performance/difficulty 
functions. 

In  the  second  phase  of  this  study,  a  new  group  of  subjects  will  be  given 
the  same  training  but  training  will  be  terminated  when  performance  reaches 
either  the  overall  or  automatism  criteria  estimates  obtained  from  the  first 
phase.  Half  of  the  subjects  will  terminate  training  when  one  criterion  is 
reached  and  half  when  the  other  criterion  is  reached.  After  the  6-month 
interval,  subjects  will  be  tested  for  retention.  Our  hypothesis  is  that 
retention  will  be  best  predicted  by  the  automatism  measure  (the  slope  of  the 
performance/difficulty  functions),  regardless  of  the  total  training  time  or  the 
overall  level  of  speed  and  accuracy  attained.  Further,  the  degree  to  which 
final  retention  performance  in  the  second  phase  matches  the  level  predicted 
from  the  first  phase  will  be  used  to  determine  the  validity  of  the  estimation 
procedure.  If  the  validity  of  this  procedure  is  established  for  this  simple 
skill,  we  will  apply  this  method  to  more  complex  skills  to  predict  the 
long-term  retention  of  the  various  components  comprising  the  skills.  Further, 
this  technique  can  be  used  to  predict  optimal  time  intervals  for  refresher 
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training. 


Calculational  word  problems .  Are  procedures  better  retained  or  more 
likely  to  appear  in  permastore  after  discovery  or  after  expository  learning? 
Which  method  of  learning  would  provide  for  better  transfer  of  knowledge  to 
novel  problems?  These  are  questions  which  address  the  long-debated  issue  of 
whether  discovery  or  expository  learning  is  better  (i.e.,  makes  information 
easier  to  retrieve  from  memory  and  to  transfer  to  new  situations).  Bruner 
(1961)  argued  that  discovery  learning  is  beneficial  for  a  variety  of  reasons, 
such  as  the  fact  that  discovery  causes  the  learner  to  organize  the  material, 
which  would  lead  to  better  retrievability  than  if  the  material  were  rote 
memorized.  Also,  discovery  would  allow  the  learner  to  form  an  hierarchically 
higher-level  conceptualization  of  the  material;  it  also  might  cause  the  learner 
to  find  alternate  solutions  to  a  problem  rather  than  merely  the  "right  way  to 
do  it,"  which  would  provide  flexibility  in  problem-solving  ability  and  would 
aid  in  transfer  to  different  situations.  Friedlander  (1965),  on  the  other 
hand,  provided  a  number  of  arguments  against  the  superiority  of  discovery 
learning.  He  noted  that  there  are  skills  and  facts  which  must  be  memorized 
before  a  larger  body  of  knowledge  can  be  learned.  Skills  and  facts,  after 
being  memorized,  may  yield  higher-level  conceptualizations.  For  example,  a 
student  may  memorize  and  use  the  equation  E  *  MC2  mechanically,  but  after  a 
period  of  time  realize  that  the  equation  means  fundamentally  that  matter  is 
energy.  Also,  discovery  learning  takes  more  time  than  does  expository 
learning,  and  the  learner  may  be  led  to  wrong  or  inappropriate  conclusions. 

McDaniel  and  Schlager  (1985)  attempted  to  determine  if  and  under  what 
conditions  discovery  learning  was  superior  to  expository  learning.  They  used 
the  domains  of  water-jar  and  river-crossing  problems.  In  these  problems,  there 
is  a  general  strategy,  which  is  an  abstract  rule  that  leads  to  problem 
solution,  and  the  procedures  which  are  the  specific  moves  that  lead  to  the 
solution.  Discovery  during  training  caused  subjects  to  solve  the  transfer 
problem  faster  than  subjects  trained  expositorily,  but  only  if  the  same  kind  of 
information  (strategy  or  procedures)  had  to  be  discovered  in  transfer  as  was 
discovered  in  training.  In  these  cases,  subjects  solved  the  transfer  problems 
faster  than  subjects  trained  expositorily,  but  they  made  the  same  number  of 
moves  to  get  to  the  solution.  McDaniel  and  Schlager  interpreted  these  results 
as  indicating  that  when  subjects  had  to  discover  procedures  or  strategies  in 
training,  they  found  which  routes  resulted  in  dead-ends,  as  well  as  those  that 
worked,  and  this  information  was  carried  over  into  the  transfer  problem.  Thus, 
discovery  subjects  learned  how  to  search  efficiently  and  where  to  stop  a 
search. 

We  intend  to  determine  if  these  results  generalize  to  other  more  complex 
and  natural  domains.  One  domain  we  would  like  to  investigate  is  that  of 
calculational  problems.  Problems  in  this  domain  have  a  general  strategy  (i.e., 
the  underlying  equation),  and  procedures  to  get  to  the  solution  (i.e.,  matching 
specific  instances  to  variables,  solving  the  equation).  We  will  first  try  to 
replicate  McDaniel  and  Schlager’ s  results  by  having  subjects  in  training 
discover  or  not  discover  strategies  or  procedures,  to  see  if  discovery  aids  in 
transfer.  We  will  then  test  their  hypothesis  that  discovery  aids  subjects  in 
finding  efficient  search  methods?  if  it  does,  then  subjects  who  go  through  the 
same  steps  (correct  steps  as  well  as  errors)  as  discovery  subjects  should  solve 
problems  as  quickly  as  the  discovery  subjects.  We  will  have  one  group  of 
subjects  solve  a  set  of  algebra  problems  (i.e.,  discover  the  solutions) 
expressed  in  simple  stories  (word  problems).  We  will  have  this  group  write 
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down  each  step,  including  searches  which  lead  to  dead-ends.  We  will  then  give 
these  procedures  to  a  second  group  of  subjects,  and  have  them  use  these 
procedures  to  solve  the  same  problems.  We  will  then  test  both  groups  on 
transfer  problems,  and  measure  their  solution  times.  If  the  only  advantage  due 
to  discovery  were  chat  of  finding  efficient  search  strategies,  then  the  groups 
should  solve  the  transfer  problems  in  the  same  amount  of  time  and  with  similar 
error  patterns.  If  discovering  the  solutions  in  training  has  some  other 
advantage,  then  the  discovery  group  should  solve  the  transfer  problems  faster 
and  more  efficiently  than  the  non-discovery  group.  We  hope  that  this  test  will 
provide  information  about  what  type  of  learning  yields  the  best  transfer  to  new 
problems.  Subsequently,  we  will  examine  the  possible  long-term  retention  of 
the  skills  involved  in  calculational  word  problems,  looking  for  components 
which  remain  specifically  a  part  of  permastore. 

Memory  for  item,  temporal,  and  spatial  information.  We  have  devised  a 
laboratory  analogue  of  the  procedural  task  involved  in  diagnosing  possible 
faults  in  an  electronic  circuit.  The  procedural  task  includes  at  least  three 
identifiable  memory  components:  the  nature  of  the  tests  themselves,  the 
temporal  sequence  in  which  the  tests  are  applied,  and  the  spatial  locations  at 
which  the  tests  are  executed.  Likewise,  the  laboratory  analogue  requires 
memory  for  three  types  of  information:  item  information  specifying  the 
identity  of  each  of  the  to-be- remembered  stimuli,  temporal  order  information 
specifying  the  time  sequence  in  which  the  stimuli  occurred,  and  spatial  order 
information  specifying  the  relative  positions  of  the  stimuli  or  the  arrangement 
of  the  stimuli  in  space. 

The  laboratory  analogue  is  based  on  an  experimental  paradigm  we  developed 
earlier  to  study  short-term  retention  (see,  e.g.,  Healy,  1974,  1975,  1977, 
1978,  1982;  Till,  Healy,  Bourne,  &  Cunningham,  1986).  In  this  paradigm, 
subjects  are  shown  a  short  list  of  letters  which  occur  one  after  another  in  a 
set  of  spatial  locations  arranged  in  a  horizontal  linear  array.  Only  one 
letter  is  shown  at  a  time  and  the  letters  do  not  usually  appear  from  left  to 
right  so  that  the  temporal  sequence  does  not  usually  correspond  to  the  spatial 
arrangement.  For  example,  the  subject  might  see  first  the  letter  B  in  the 
middle  of  three  positions,  then  the  letter  S  in  the  right-most  position,  and 
last  the  letter  K  in  the  left-most  position.  The  letter  identities  are  B,  K, 
and  S;  the  temporal  sequence  is  BSK;  and  the  spatial  arrangement  is  KBS.  Three 
different  conditions  are  employed  in  order  to  examine  separately  memory  for  the 
three  kinds  of  information.  in  the  item  condition,  the  temporal-spatial 
arrangement  of  the  letters  does  not  vary  across  trials  (e.g.,  the  first  letter 
might  always  occur  in  the  middle  position,  the  second  letter  in  the  right-most 
position,  and  the  third  letter  in  the  left-most  position,  as  in  the  earlier 
example);  only  the  identities  of  the  letters  change  from  trial  to  trial.  For 
example,  the  letter  identities  may  be  B,  K,  and  S  on  one  trial,  and  M,  R,  and  s 
on  another  trial.  In  contrast,  in  the  temporal  condition,  the  letter 
identities  are  fixed  across  trials,  as  is  the  spatial  arrangement  of  the 
letters  (e.g.,  the  letters  shown  might  always  be  B,  K,  and  S,  with  B  in  the 
middle,  S  on  the  right,  and  K  on  the  left);  only  the  temporal  sequence  chances 
from  trial  to  trial.  For  example,  the  temporal  sequence  may  be  BSK  on  one 
trial  and  BKS  on  another  trial.  Finally,  in  the  spatial  condition,  the  letter 
identities  are  fixed  across  trials,  as  is  their  temporal  sequence  (e.g.,  the 
letters  shown  might  always  be  B,  K,  .and  S,  with  B  first,  S  second,  and  K 
third);  only  the  spatial  arrangement  changes  from  trial  to  trial.  For  example, 
the  spatial  arrangement  might  be  KBS  on  one  trial  and  KSB  on  another. 
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With  item,  temporal,  and  spatial  information  separated  in  this  way,  a 
number  of  crucial  differences  have  been  found  in  the  characteristics  of 
short-term  retention  for  the  three  types  of  information.  For  example,  Healy 
'1975)  found  evidence  for  phonemic  coding  and  a  steep  time  course  of  forgetting 
in  temporal  recall  bur.  nor  in  sparial  recall.  Further,  more  recently,  Till  et 
ai.  1 1986 )  found  evidence  for  age  deficits  in  spatiai  recall  but  ncr  in  icem 
recall,  with  temporal  recall  showing  intermediate  age  effects.  It  is  iikeiy 
that  the  three  types  of  information  will  also  show  important  differences  in 
their  long-term  retention  and  maintenance. 

In  order  to  study  long-term  retention  and  maintenance  of  information,  the 
paradigm  devised  to  study  short-term  retention  will  be  modified  in  several 
ways.  First,  and  most  crucially,  longer  retention  intervals  will  be  used.  The 
previous  short-term  recall  experiments  involved  retention  intervals  varying 
from  0  to  30  seconds;  the  present  investigation  will  involve  study-test 
retention  intervals  ranging  from  30  seconds  to  30  minutes  and  test-retest 
maintenance  intervals  ranging  from  1  day  to  1  year.  Second,  longer  lists  will 
be  used  and  a  correspondingly  greater  number  of  spatial  positions.  In  the 
previous  studies  of  short-term  retention,  list  lengths  varied  from  three  to 
five  items,  whereas  in  the  present  case,  we  propose  to  employ  lists  20  items  in 
length.  Third,  instead  of  using  letters  as  to-be- remembered  items,  words  wiil 
be  employed,  so  that  the  stimuli  will  be  more  meaningful  than  those  used 
previously.  Fourth,  the  fixed  order  of  the  words  (the  temporal  sequence  in  the 
spatial  condition,  the  spatial  arrangement  in  the  temporal  condition,  and  both 
the  temporal  and  spatial  orders  in  the  item  condition)  will  always  correspond 
to  the  alphabetical  order,  to  ensure  that  the  fixed  order  will  not  be  forgotten 
by  the  subjects  (see,  Healy,  1977,  for  a  similar  manipulation).  More 
specifically,  the  subjects  will  be  given  lists  of  words,  each  of  which  begins 
with  a  different  one  of  the  20  consonants  in  the  alphabet  (a,  e,  i,  o,  u,  and  y 
are  excluded).  For  example,  a  typical  list  might  be:  ballot,  charter,  deacon, 
fairy,  gesture,  hiccup,  javelin,  kitchen,  leopard,  motion,  newsprint,  pilgrim, 
quarry,  rubbish,  silver,  table,  virtue,  window,  xenon,  and  zombi .  In  the  item 
condition,  subjects  must  learn  and  remember  the  specific  words  shown  on  a  trial 
but  will  know  in  advance  that  the  words  will  be  presented  in  alphabetical 
temporal  sequence  and  alphabetical  spatial  arrangement.  In  the  temporal 
condition,  the  subjects  will  know  in  advance  which  words  will  be  shown  and  that 
the  words  will  be  arranged  in  an  alphabetical  spatial  arrangement;  however, 
they  must  learn  and  remember  the  temporal  sequence  of  the  words.  In  the 
spatial  condition,  the  subjects  will  know  in  advance  which  words  will  be  shown 
and  that  the  words  will  be  displayed  in  an  alphabetical  temporal  sequence,  but 
they  must  learn  and  remember  the  spatial  arrangement  of  the  words.  In  each 
condition,  subjects  will  learn  a  single  list  of  words  to  the  criterion  of  two 
successive  correct  recall  attempts. 

We  expect  that  the  long-term  retention  characteristics  of  the  three  types 
of  information  will  vary,  and  we  suspect  that  the  optimal  conditions  for 
maintaining  the  three  types  of  information  will  vary,  but  it  is  not  clear 
precisely  what  form  the  variation  will  take.  Perhaps,  for  example,  as  in  the 
case  of  short-term  retention,  temporal  information  will  shew  a  steep  time 
course  of  forgetting.  But  it  may  be  that  phonemic  coding  will  not  provide  the 
basis  for  long-term  temporal  order  recall,  in  which  case  the  time  course  of 
forgetting  may  be  no  greater  for  temporal  than  for  spatial  information. 

Dual  task.  The  dual-task  paradigm  (e.g. ,  Posner  &  Boies,  1971)  has  been 
used  to  assess  attentional  demands  and  the  attainment  of  automatism  during 
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skill  acquisition  (see,  e.g.,  Spelke,  Hirst,  &  Neisser,  1976).'  Likewise,  we 
have  developed  a  dual-task  paradigm  to  evaluate  the  attentional  requirements  of 
a  skill  during  its  acquisition,  retention,  and  maintenance  phases.  We  have 
elected,  for  our  initial  studies,  to  investigate  the  motor  skills  involved  in 
learning  to  type  numbers  on  the  keypad  of  a  computer  terminal.  The  primary 
task  will  thus  be  keying-in  a  three-digit  number  displayed  on  the  computer 
terminal  screen,  and  response  latency  will  be  the  major  measure  of  primary  task 
performance,  assuming  that  response  accuracy  is  at  or  close  to  the  ceiling  of 
100%  correct.  The  secondary  task  we  have  chosen  is  word  naming,  which  is  known 
to  be  automatic  for  adult  subjects.  More  specifically,  three-letter  common 
words  will  be  shown  to  the  subjects  immediately  below  the  three-digit  numbers 
on  the  computer  screen,  and  the  subjects  will  be  requested  to  name  the  words 
aloud.  Naming  latency  will  be  used  as  the  major  measure  of  secondary  task 
performance,  again  assuming  the  error  rate  is  minimal. 

In  order  to  assess  the  effects  of  changing  and  variable  response  patterns 
on  the  acquisition  and  subsequent  retention  and  maintenance  of  the  number 
keying  skill,  we  will  compare  three  different  training  conditions.  In  the 
constant  response  pattern  condition,  the  nine  digits  will  occur  in  only  three 
different  three-digit  patterns  (e.g.,  158,  629,  473).  Thus,  every  time  a  given 
digit  (e.g.,  5)  is  to  be  keyed,  it  will  always  be  in  the  same  response  context 
(immediately  preceded  by  1  and  immediately  followed  by  8).  In  contrast,  in  the 
intermediate,  or  semi -variable,  response  pattern  condition,  the  nine  digits 
will  occur  in  only  three  different  three-digit  combinations,  but  the 
permutations  of  the  digits  will  vary.  Therefore,  six  different  response 
patterns  will  be  used  for  a  single  three-digit  combination  (e.g.,  158,  185, 
518,  581,  815,  851).  Thus,  in  the  semi -variable  condition,  every  time  a  given 
digit  (e.g.,  5)  is  to  be  keyed,  it  will  always  be  accompanied  by  the  same  pair 
of  digits  (1  and  8),  but  the  sequence  of  the  three  digits  will  vary  across 
trials.  Finally,  in  the  fully  variable  response  pattern  condition,  the  nine 
digits  to  be  keyed  will  occur  in  all  possible  combinations  and  permutations 
(e.g.,  158,  185,  562,  785,  etc.).  Thus,  in  this  condition,  the  context  in 
which  a  given  digit  (e.g.,  5)  occurs  is  unpredictable  and  changing  from  trial 
to  trial.  Each  of  these  response  pattern  conditions  will  occur  both  in 
dual-task  format,  with  a  three-letter  word  accompanying  each  three-digit 
number,  and  in  single-task  or  control  format,  with  no  words  accompanying  the 
digits.  Each  subject  will  participate  in  only  one  training  condition  under 
both  dual-task  and  single-task  formats.  Attentional  demands  of  the  three 
conditions  will  be  assessed  in  part  by  comparing  primary  task  latency  under  the 
two  formats  (with  or  without  the  secondary  task).  One  defining  characteristic 
of  automatism  will  be  that  the  response  latency  on  the  primary  task  is 
comparable  under  the  two  formats.  Further,  subjects  will  participate  in  a 
baseline  condition  involving  a  secondary  task  which  also  consists  of  naming 
three-letter  words.  The  second  defining  characteristic  of  automatism  will  be 
that  naming  latency  obtained  in  the  baseline  condition  be  comparable  to  that 
obtained  as  the  secondary  task  variable  in  the  dual-task  format  . of  the 
experimental  conditions. 

Subjects  will  be  trained  over  a  period  of  8  weeks,  with  two  1-hour 
training  sessions  each  week.  Retention  will  then  be  tested  after  2,  4,  and  6 
months,  with  no  practice  during  the  maintenance  intervals. 

Following  the  observations  of  Shiffrin  and  Schneider  (1977)  concerning  the 
differences  between  constant  and  variable  mapping  in  the  acquisition  of 
automatic  target-detection  performance,  we  expect  that  automatism  will  be 
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achieved  most  rapidly  in  the  constant  response  pattern  condition  and  that 
automatism  will  not  be  achieved  in  the  fully  variable  response  pattern 
condition;  the  predictions  for  the  semi -variable  condition  are  less  clear, 
although  performance  is  expected  to  be  intermediate  for  that  condition.  In 
contrast,  retention  performance  is  expected  to  show  the  reverse  pattern.  In 
accordance  with  Schmidt's  (1975)  schema  theory,  we  expect  that  after  a  long 
time  interval  without  skill  practice,  performance  will  be  better  under  the 
fully  variable  or  at  least  under  the  semi -variable  response  pattern  condition 
than  under  the  constant  response  pattern  condition.  This  prediction  that 
variability  will  enhance  skill  maintenance  is  also  consistent  with  Battig's 
(1979)  intratask  interference  principle. 

Generation  tasks.  A  number  of  experiments  have  demonstrated  that 
individuals  remember  material  better  by  generating  it  themselves  rather  than 
simply  reading  the  same  material  when  it  is  presented  to  them  by  an 
experimenter.  In  many  of  these  experiments,  the  stimuli  are  pairs  of  words 
presented  to  subjects  under  two  conditions:  a  read  condition  and  a  generate 
condition.  In  the  read  condition,  a  pair  of  words  is  presented  and  subjects 
read  the  pair  aloud.  In  the  generate  condition,  a  word  pair  is  presented  with 
the  first  word  intact  and  the  second  missing  one  or  more  letters;  subjects  must 
then  generate  the  second  word  of  the  pair  using  the  first  word  as  a  context. 
The  generation  effect,  as  it  has  been  called,  has  been  demonstrated  for  a 
variety  of  materials,  including:  words  (Donaldson  &  Bass,  1980;  Glisky  & 
Rabinowitz,  1985;  Jacoby,  1978;  McFarland,  Frey,  &  Rhodes,  1980;  Nairne,  Pusen, 
&  Widner,  1985;  Slamecka  &  Graf,  1978),  sentences  (Graf,  1980),  meaningful 
bigrams  (Gardiner  &  Hampton,  1985),  and  numbers  (Gardiner  &  Hampton,  1985; 
Gardiner  &  Rowley,  1984);  and  the  effect  has  been  obtained  using  semantic, 
orthographic,  rhyming,  and  other  generate  rules. 

Along  with  other  experimenters  (e.g.,  Nairne  &  Widner,  1987),  we 
believe  that  whether  one  obtains  a  generation  effect  or  not  depends  on  what  is 
generated,  the  type  of  rule  used  to  generate  it,  and  the  type  of  test  used  to 
assess  retention.  Gardiner  and  Rowley  (1984),  for  example,  found  that  when 
subjects  generated  answers  to  multiplication  problems  instead  of  reading 
answers  supplied  by  an  experimenter,  subjects  recalled  the  answers  much  better. 
However,  in  an  experiment  of  our  own  (Crutcher  &  Healy,  1986),  we  found  that 
although  mentally  generating  an  answer  resulted  in  superior  retention  compared 
to  simply  reading  an  answer  supplied  by  the  experimenter,  generating  the  answer 
with  a  calculator  did  not  significantly  improve  retention  over  simply  reading 
the  problem  and  answer.  Furthermore,  when  subjects  were  presented  with  a 
correct  answer  but  were  told  to  verify  whether  the  answer  was  correct  or  not, 
retention  performance  was  comparable  to  when  subjects  had  mentally  generated  an 
answer. 

These  results  suggest  that  the  mental  operations  performed  when  internally 
generating  the  multiplication  products  were  different  from  those  performed 
during  external  generation  (i.e.,  using  the  calculator).  Presumably,  the 
subiects  who  internally  generated  the  answers  had  to  multiply  the  two  numbers 
together  mentally  or  at  least  consult  their  knowledge  of  the  multiplication 
facts.  Later,  during  recall,  they  could  use  these  mental  operations  as  cues  to 
help  retrieve  the  answers.  The  verbal  protocols  which  we  collected  from  half 
of  the  subjects  support  this  hypothesis.  In  most  of  the  correctly  recalled 
answers,  subjects  recalled  the  entire  problem  (e.g.,  3x6-  18)  or  recalled 
the  two  multipliers  and  then  used  them  to  cue  their  recall  of  the  answer. 
Though  it  is  not  clear  that  verifying  answers  involves  the  same  mental 
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operations  as  generating  the  answers  (because  the  answer  is  supplied,  it  may 
serve  as  a  priming  cue),  apparently  the  operations  that  occur  during 
verification  are  also  useful  as  retrieval  cues.  Producing  the  answers  with  a 
calculator  required  selecting  and  pushing  the  correct  keys  and  reading  the 
answer  off  the  display,  but,  at  test,  the  mental  operations  associated  with 
these  tasks  were  apparently  not  useful  in  attempting  to  recall  the  answers. 
Given  a  different  sort  of  retention  test,  one  that  specifically  tapped  the 
subjects'  memory  for  these  operations,  one  would  more  likely  obtain  a  retention 
advantage  for  the  problems  performed  on  the  calculator.  These  results  support 
our  hypothesis  that  the  retention  benefits  associated  with  generating  material 
oneself  depend  on:  what  material  is  actually  generated,  how  the  material  is 
generated,  and  how  retention  is  assessed. 

Given  the  size  and  robustness  of  the  generation  effect,  we  propose  to  use 
the  generation  paradigm  as  a  methodology  to  examine  what  types  of 
learner-generated  material  and  rules  will  lead  to  superior  retention  under 
differing  test  conditions.  We  suspect  that  material  which  is  self-generated 
may  often  be  highly  resistant  to  decay.  However,  because  no  experiments  have 
tested  whether  the  retention  advantage  demonstrated  for  self-generated  material 
will  extend  to  longer  retention  intervals,  we  plan  to  test  this  hypothesis  in 
the  near  future.  We  then  propose  to  extend  the  generation  paradigm  to  examine 
the  effects  of  self-generation  for  material  that  would  likely  be  encountered  in 
skill  acquisition  settings.  If  this  paradigm  can  be  extended  to  tasks  other 
than  simple  word  completion  and  arithmetic  tasks,  it  holds  promise  of  being  a 
useful  methodological  tool  for  assessing  what  types  of  generation  tasks  lead  to 
optimal  long-term  retention.  Tasks  we  are  considering  looking  at  include:  (a) 
having  learners  generate  the  reasons  for  performing  specific  task  steps  instead 
of  supplying  them  with  the  reasons,  (b)  having  learners  solve  problems  for 
themselves  instead  of  providing  them  with  answers,  and  (c)  asking  learners  to 
write  down  in  their  own  words  definitions  for  new  vocabulary  terms  rather  than 
simply  giving  them  the  definitions. 

Previously  learned  skills.  In  order  for  previously  learned  skills  to  be 
tested  and  re-trained,  a  measure  of  the  subjects'  initial  training  and  later 
retention  of  the  skill  must  be  obtained.  Examples  of  previously  learned  skills 
are  multiplication,  finding  square  roots,  and  solving  algebra  equations.  A 
questionnaire,  similar  to  that  used  by  Bahrick  (1984),  will  be  used  to  assess 
when,  how,  and  to  what  extent  a  subject  was  originally  trained  in  a  skill.  The 
questionnaire  will  also  assess  the  type  and  amount  of  maintenance  that  the 
skill  received  after  the  original  training.  The  general  format  of  the 
questionnaire  will  be  as  follows: 

Original  Training: 

1.  When  was  the  skill  first  learned? 

2.  What  was  the  performance  level  immediately  after  the  original  training 
was  finished? 

3.  What  strategies  were  used  to  memorize  the  information  or  operations? 

4.  What  strategies  were  used  to  perform  the  operations? 

5.  Approximately  how  quickly  could  the  operations  be  performed? 
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Retention  and  Estimates  of  Current  Performance: 

1.  How  much  rehearsal  did  the  skill  receive? 

a.  What  classes  or  other  training  were  taken  which  involved  the  use  of 
these  operations  or  this  information? 

b.  What  everyday  activities  were  performed  which  involved  the  use  of 
these  operations  or  this  information? 

2.  What  strategies,  if  any,  are  currently  used  to  perform  these 
operations? 

3.  Approximately  what  is  the  current  performance  level?  (The  actual 
performance  level  will  be  measured  and  may  differ  from  the  subject's 
estimate  of  his  or  her  ability.) 

A  sample  questionnaire  which  assesses  the  skill  of  multiplication  is  shown 
below: 

Original  Training: 

1.  When  did  you  first  learn  the  multiplication  tables  (what  year  and  what 
grade  in  school)? 

2.  To  what  level  did  you  memorize  the  multiplication  table? 

Never - Up  to - Up  to - Up  to - Up  to - Up  to - Up  to - Over 

Memorized  5x5  9x9  10x10  11x11  12x12  13x13  13x13 

3.  What  strategies  did  you  use  to  learn  the  table  (e.g.,  rote 
memorization,  reference  to  addition)? 

4.  what  strategies  did  you  use  when  you  had  to  compute  the  answer  to  a 
multiplication  problem? 

5.  Estimate  how  quickly  you  were  able  to  answer  the  following  problems 
when  you  completed  your  first  year  of  learning  multiplication: 

0-5  sec  -  5-10  sec  -  10-20  sec  -  20-30  sec  -  30-45  sec  -  45  sec-1  min. 

a.  5x5 

b.  7x6 

c.  9x2 

d.  6x4 

Retention  and  Estimates  of  Current  Performance: 

1.  what  math  courses  have  you  had  since  you  learned  the  multiplication 
table? 

Grade  School  Middle  School  High  School  College 
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2.  What  other  math-related  courses  have  you  had  since  you  learned  the 
multiplication  table  (e.g.,  physical  sciences,  engineering, 
statistics)? 

Grade  School  Middle  School  High  School  College 


3 .  How  often  do  you  use  a  calculator? 

Over  once  a  week  -  Over  once  a  month  -  Over  once  a  year  -  Never 

4.  How  often  do  you  use  paper  and  pencil  to  do  computations? 

Over  once  a  week  —  Over  once  a  month  -  Over  once  a  year  -  Never 

5.  How  often  do  you  do  computations  in  your  head? 

Over  once  a  week  —  Over  once  a  month  -  Over  once  a  year  -  Never 

6.  List  the  everyday  activities  in  which  you  use  multiplication,  how 
often  you  engage  in  these  activities,  and  how  you  normally  calculate 
the  answers: 

Method 

of  Calculation 
(calculator, 
paper  &  pencil, 
or  in  head) 


a .  shopping 

b.  cooking 

c.  calculating  areas 
(e.g.,  wallpaper, 

rugs) 

d.  other  (please  list) 

7.  what  strategies  (if  any)  do  you  use  now  to  solve  multiplication 
problems? 

8.  To  what  level  do  you  think  you  know  the  multiplication  table  now 
(without  computing  the  answers)? 

Bahrick  (1984)  used  the  questionnaire  he  developed  to  show  what  factors 
influence  the  retention  of  Spanish.  He  found  that  factors  involved  in  original 
training  predicted  retention:  Higher  grade  received  acid  higher  level  of 
training  (in  Spanish  and  in  other  Romance  languages)  indicated  better 
performance  on  various  tests  of  Spanish  knowledge.  The  amount  of  time  since 
original  training  also  influenced  retention;  shorter  retention  intervals  were 
associated  with  better  retention.  The  above  general  questionnaire  is  predicted 
to  indicate  the  same  effects  of  training  factors  on  retention:  Higher 
performance  at  the  end  of  training,  higher  levels  of  training,  and  a  shorter 
interval  between  training  and  test  should  lead  to  better  performance  on  a 


Activity 


How  often  (e.g. , 
once  per  week ) 


For  how 
many  years 


11 


retention  test  of  the  relevant  skill. 

Bahrick  also  found  that  rehearsal  factors  did  not  predict  performance  on 
the  retention  tests.  He  stated  that  this  may  be  due  to  the  fact  that  most 
people  rehearsed  Spanish  very  little  since  they  were  originally  trained.  The 
rehearsal  factors  on  the  above  questionnaire  will  be  used  to  determine  if 
rehearsal  has  any  effect  on  retention  of  a  skill.  It  seems  likely  that 
rehearsal  will  improve  retention,  and  also  that  some  types  of  rehearsal  will 
aid  retention  more  than  others.  For  example,  subjects  who  compute 
multiplication  problems  in  their  heads  should  be  faster  and  more  accurate  than 
subjects  who  have  mainly  used  a  calculator  for  computations  since  original 
training  in  multiplication. 
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